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ABSTRACT 



Prepared for those interested in program evaluation, 
this bibliography (largely annotated) on management development and • 
training contains 61 items published between 1959 and early 1969. 
Citations have been grouped into five categories: descriptions of 
research studies; literature reviews; discussion of evaluation 
techniques; surveys of corporate practices; and books (eight 
references) . Such aspects as participant satisfaction, interpersonal 
competence, sensitivity and human relations training, program 
Dlanning, behavior and attitude change, reinforcement, and the 
Management Grid approach are represented. Topics not covered include 
evaluation of undergraduate management courses or manager training 
programs conducted under university auspices; assessments of 
programed instruction as a management development technique; and 
military research. An author index and a list of periodicals are 
included, (ly) 
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A SELECTED BIBLIOGRAPHY 
ON 

EVALUATION OP MANAGEMENT TRAINING AND DEVELOPMENT PROGRAMS 



Compiled and Annotated 

by 

Vera Kohn 



This bibliography was prepared as an aid to those interested in the evaluation 
of management training and development programs. It includes material 
published between 1959 and the early part of 1969. 



The citations have been grouped into five categories: 



I. 

II. 

III. 

IV. 

V. 



Descriptions of Research Studies 
Reviews of the Literature 
Discussions of Evaluation Techniques 
Surveys of Corporate Practices 
Books 



The following topics are not covered in this bibliography: evaluation of 

undergraduate management courses or manager training programs conducted under 
the auspices of universities; assessments of the effectiveness of programed 
instruction as a management development technique; and research sponsored by 
the military. 



An author index and a list of the periodicals which were canvassed appear at the 
end of the bibliography. 



Additional copies may be obtained from the American Foundation for Management 
Research at $ 1.50 per copy. 
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A SELECTED BIBLIOGRAPHY ON EVALUATION OF MANAGEMENT TRAINING & DEVELOPMENT PROGRAMS 



1 

A 






I. Descriptions of Research Studies 

The references in this section have been classified according to the vantage point 
from which the researcher assessed the impact of the management training and 
development program. A study may be listed under more than one heading. 

Reaction 

* "How satisfied was the participant with the program?" 

1. House, Robert J. , "Managerial Reactions to Two Methods of Management Training," 
Personnel Psychology , Autumn PP» 3H“319. 

The study compares managerial reactions to leader -centered and 
student -centered methods of training in a 4-week (l60 hours) 
management development program and reports managers’ opinions 
concerning optimum teaching methods. The sample consisted of 
43 managers. At the conclusion of the program, they filled 
out structured and open-ended questionnaires designed to tap 
their reactions to the types of instruction under investigation. 

The chi-square test was applied to determine whether reactions 
to the student -centered method were significantly different 
from reactions to the leader -centered technique. 
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The results showed no clear superiority of either method for 
purposes of gaining participants’ enthusiasm or holding their 
attention. 

2. Kohn, Vera, An Assessment of Participants’ Reactions to Management Development 
Meetings . New York: American Foundation for Management Research, Inc., 196b, 
67 PP. 



% 



This report describes a study designed to (l) develop, and 
test the validity of, a research instrument measuring parti- 
cipants’ feelings about management development programs they 
attended; (2) explore the relationship between feelings about 
selected aspects of the learning situation and satisfaction 
at outcome; (3) determ ine whether "correlated of satisfaction 
vary with the format of the program. The research was carried 
out at two of the several types of meetings offered by the 
American Management Association, the Workshop Seminar 
(instruction based on discussion) and the Orientation Seminar 
(instruction based on lectures). The sample consisted of 
2,000 participants, 1,000 from each meeting-type. Data were 
collected by means of questionnaires completed by attendees 
at the conclusion of each meeting. The principal statistical 
tool was the multiple regression analysis. 
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The results showed that the most important contributors to 
satisfaction at outcome were: (l) Subject matter that has 
practical value (both meet ing -types ) ; (2) balance of background 
characteristics (e.g. , experience, organizational level, 
company size, type of business) to assure meaningful communica- 
tion among the learners (Workshop Seminar only); (3) opportu- 
nity for learner participation (Orientation Seminar only). 

Thus, "correlates” of satisfaction were found to vary with 
program format. 




Kohn, Vera, and Treadway C. Parker, Management De velopment and Program 
Evaluation: Partners in Promoting Managerial Effectiveness. New York: 
American Foundation for Management Research, Inc., 1969 , 124 pp. 



This report describes a study designed to evaluate the impact 
of the American Management Association's 4-week Management 
Course on participants from a large company. The sample was 
composed of 37 managers, 27 of whom furnished data while they 
were enrolled in the Course; the other 10 were Course graduates 
who had taken the program a number of years before the study 
was initiated. Measuring devices included (l). questionnaires 
and "Time Distribution Profiles," administered on a pre- and 
post -training basis; (2) notebooks in which respondents 
recorded their impressions while attending the program; 

(3) group interviews at the conclusion of each week of the 
Course; (4) individual interviews with each Course graduate. 

The authors found statistically significant changes in atti- 
tude toward various aspects of the managerial role. Participants' 
reactions to specific features of the learning situation 
(program content, presentation, methods of instruction, etc.) 
are presented in detail, and recommendations calculated to 
improve program design are included. The application- of both 
findings and research approach to other management development 
programs is discussed. 

4. Moffie, Dannie J. , Richard Calhoon, and James K. O'Brien, "Emluation of a 

Management Development Program." Personnel Psychology, Winter 1964 ) pp. 431-440. 

The purpose of the study was to evaluate a training course 
in problem-solving and decision-making given to three 
levels of management at a large southeastern paper mill. 

Approximately 50 management personnel attended the 20 -hour 
course. The research design included (l) course-end question- 
naire to get the reactions of participants; (2) a controlled 
experiment, involving two tests, to evaluate the effects 
of training on learning; (3) analysis of observational data 
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collected by two observers during "practical work periods" 
of the training program. 



The results showed that trainees considered the course 
worthwhile and applicable to their work. The program did 
not result in significant improvement in scores on test 
devices used nor did the observational data reveal any 
substantial changes. 



i 



i 



Morton, Robert B. , and Bernard M. Bass, "The Organizational Training Laboratory, 
Training Directors Journal, October 19o4, pp. 2-15. 



1 




This article describes an organizational training laboratory 
for managers from different levels in the same department 
of an aerospace corporation. Three evaluative approaches 
were used. The 107 trainees were asked (l) to indicate satis- 
faction with the program on a series of scales ( post -training) ; 
(2) to complete questions relating to concern for boss, peers, 
subordinates, rules of the organization, etc., in significant 
management areas both in terms of how much they were actually 
concerned and how much they felt they ought to be concerned 
(pre- and post -training) ; (3) to report any critical job 
Incidents which had occurred since training and which they 
considered a consequence of training (three months after 
completion of the program). 
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The data revealed that (l) on average, level of satisfaction 
with the program was near the high end of the 9 -point scale on 
most of the questionnaire items; (2) after training, there was 
a marked increase in concern for problems of intraorganizational 
decisions, conflict resolution, coordination, etc.; nevertheless, 
respondents felt they were less concerned than they ought to be; 
(3) most of the incidents reported by 97 of the original trainees 
dealt with personal improvement, improved work relationships, 
and improved organizational climate. 



Si 



O'Donovan, Thomas R. , "The Climate Approach to Management Development," Management 
of Personnel Quarterly . Summer pp. 28-31. 



The objective of the study was to determine if trainee 
evaluations of management development programs differentiated 
successful from unsuccessful programs. The author conducted 
interviews with ex -participants of management development 
programs discontinued by the firms administering them because 
they were judged as inadequate for the purpose of developing 
managers. Most of the respondents were no longer with the 
company whose training program they had taken. The same 
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questions were then asked of participants of allegedly 
successful management development programs. 



The findings were that the problems seen by the second 
sample were not very different from the problems identified 
by the first group. The author argues that companies must 
provide a "corporate climate" conducive to s elf -development , 
and that this will facilitate management development more 
than formal training programs. 



1 ■ 




rG SOC ’ J ° h ” E, > Donald P. 'Mullen, and Robert Morton, "Sensitivity Training for 
Individual^ Growth - Team Training for Organization Development?" Tr ainin g and 
Development Journal . January 1968, pp. 47 - 53 . “ 



This study compares the reactions of participants in a 
sensitivity training program with the reactions of participants 
m an organization training, laboratory program. Managers were 
asked to assess the value of the training experience to them- 
selves (1) as individuals y and (2) as managers in their 
organizations by giving scaled responses to specific questions. 
In addition, they reported actions they were taking, after 
training, that they had not taken before as well as actions 
they were no longer taking that they had taken before. 



On the basis of the responses, the authors concluded that the 
two types of training were equally valuable for the individual 
m his interpersonal relationships, both on and off the job 
but that the organizational team training approach was more* 
effective for the trainee in his role as manager in an organization, 



Learning 



What attitudes, techniques, or skills were 
understood and absorbed by the participant?" 



Abbatiello, Aurelius A., "An Objective Evaluation of Attitude Change in Training," 
Trainin g and Development Journal . November 1967, pp. 23 - 34 . S 



• ^ Ur ^ 0S 6 was to determine whether changes in 

attitudes occurred as a result of participation in a 17-day 
training program on supervisory practices . Trainees were 
41 supervisors in three program groups. The research 
instrument was a word-association test requiring subjects to 
rate a given concept on a graphic scale. Scales covered three 
major factor loadings - Evaluation (e.g., good-bad), Activity 
\ ® • S • j active -pas s ive ) ,, and Potency (e.g., weak-strong). 
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Twelve concepts were selected for study* The tests were 
administered before and after the program. 



The researcher concluded that change in attitude took place; 
that the direction of change was from polarity toward the 
neutral point on the scale; and that of the three factor 
loadings, Evaluation was the most sensitive indicator of change. 



9. 




Bass, Bernard M. , '’Reactions to Twelve Angry Men as a Measure of Sensitivity 
Training," Journal of Applied Psychology , April 1962, pp. 120-124. 

The purpose of the study was to assess the impact of a 
management training laboratory on participants’ perceptions 
of interpersonal relationships. The sample consisted of 
34 executives who were shown the film, Twelve Angry Men, 
before and after two weeks of T-group training. After each 
showing of the film, they were asked to finish a series of 
incomplete sentences describing the behavior of the 
characters portrayed in the film. 



The data indicated that, after training, participants had 
become significantly (p<. 0 l) more sensitive to the inter- 
personal relationships among the characters. 



To eliminate the possibility that merely seeing the fi lm 
twice had improved performance on the test, two other groups 
of trainees were shown the film only once, post -training. 

Similar results were obtained. The author concluded that 
the increased sensitivity to interpersonal relations had been 
due to the training experience and not due to seeing the 
film twice. 

10. /Baum, Bernard H. , Peter F. Sorensen, Jr., and William S. Place, "Organizational 
V Effects of Supervisory Human Relations Training: An Evaluative Technique," 

Personnel Journal . Mar ch t 1966 , pp. 148-152. 



This article deals with an exploratory study designed to 
.determine the feasibility of using distribution of influence 
as a measure of change as a result of supervisory training. 
Respondents were asked to indicate their perception of the 
amount of influence (l) exercised by each of the hierarchical 
levels in their departments; ( 2 ) they feel they themselves have; 
(3) they think they should have. Questionnaires were 
distributed to l4 trainees, their supervisors and subordinates 
before and three months after training. 



Partial findings are reported for only one of the fourteen 
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departments. These are: total influence of the department 

■was seen as increasing as a result of training. Subordinates 
were perceived as gaining influence. The participant viewed 
his own influence as constant although his subordinates 
perceived it as decreasing. Before the program, the participant 
desired more influence but afterwards he felt that he had 
enough influence. 

11. Blake, Robert R. , and Jane S. Mouton, "Some Effects of Managerial Grid Seminar 
Training on Union and Management Attitudes Toward Supervision, " Journal of 
Applied Behavioral Science , Oetober/Novembor /December 1966, pp . 387 -400 * 

Thirty-three managers and 23 union members from the same 
plant participated in two 1-wsek Managerial Grid Seminars. 

A forced -choice questionnaire, administered before and after 
training, assessed beliefs about supervisory practices. 

The results showed (l) significant differences in beliefs 
between management and union personnel as to what constitutes 
sound supervision; (2) both union and management attitudes 
changed as a result of the Seminar; the change was toward 
increased endorsement of a style of supervision emphasizing 
production; (3) managements attitude changed more than that 
of the union members. 

12. Carron, T. J. , "Human Relations Training and Attitude Change: A Vector Analysis," 

Personnel Psychology, Winter 19$D PP. 403-424. 

The author conducted an experimental field study to determine 
the effect of T -group training on the opinions of Research 
and Development managers toward two kinds of leader behavior. 

Participants completed a questionnaire before and after 
training. Responses were compared with those of an untrained 
control group. 

* 

Carron found that a significantly larger number of trained 
than untrained changed their opinions concerning ideal leader 
behavior; after training, they placed higher value on 
consideration and less value on structure. 

13. Harrison, Roger, "Cognitive Change and Participation in a Sensitivity Training 
Laboratory," Journal of Consulting Psychology, December 1966, pp. 53-7 “520. 

This study was designed to discover whether participants in a 
sensitivity training laboratory would become more oriented to 
interpersonal and intrapersonal processes after training and 
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whether the change would be related to rating of participants * 
behavior during training. The sample consisted of 115 trainees. 

They described co-workers before and after training, using a 
modified form of Kelly's Role Repertory Test. Concepts were 
classified as inferential-expressive and concrete -instrumental. 

Each participant was also rated on his behavior by the others 
in his laboratory. 

Significant changes were found toward use of a greater pro- 
portion of concepts dealing with feelings, attitudes, and 
emotions (inferential-expressive) in describing others. Changes 
.were slight three weeks after training but increased to signi- 
ficance after three months. Significant positive correlations 
were found between change in concept usage and rating of active 
involvement in the training process. The author feels that 
the evidence justifies the conclusion that interpersonal 
perceptions are amenable to change through sensitivity training 
and that the change is associated with the extent to which the 
trainee was involved in the learning situation. 

14. House, Robert J., and Henry Tosi, "An Experimental Evaluation of a Management 
Training Program," Academy of Management Journal , December PP» 303"315* 

This study was designed to test the importance of "climate" to 
the effectiveness of a management development program and to 
determine the relationship between certain pre-training 
characteristics and change resulting from the training effort. 

A course consisting of lectures, readings, and discussion 
was administered to five levels of management at the 
Engineering Division of a company. Top level management was 
trained before subordinate groups. Scaled questionnaires 
were completed by the trainees, their subordinates, and their 
superiors immediately prior to and up to 18 months after 
training. The questionnaires were administered at the same 
time to an untrained control group consisting of members of 
another department. Data for 2k trained and 33 untrained 
personnel were analyzed. 

The findings showed that the training did not result in 
greater gross changes in the trained group than in the untrained 
group, suggesting that while compatible climate is a necessary 
condition, it is not a "sufficient" condition for the success 
of a management development effort. However, significant 
differences were noted between the trained and untrained groups 
with respect to certain pre-braining characteristics. Data 
indicated that the persons who are most likely to change their 
perceptions and behavior are those who are satisfied with 
their positions, feel relatively secure in their jobs, and 
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perceive themselves as having a relatively high amount of 
authority. The authors conclude that compatible climate plus 
participant characteristics constitute a "sufficient" condition 
for change. 



15 . 



Kohn, Vera, and Treadway C. Parker, Management Development and Program Evaluation: 
Partners in Promoting Managerial Effectiveness. New York: American Foundation 
for Management Research, Inc., I 969, 124 pp. 



*2 
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For abstract, see item 3* 
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16 . 



Lawshe, C. H., Robert A. Bolda, and R. L. Brune, "Studies in Management Training 
Evaluation: II. The Effects of Exposures to Role Playing," Journal of Applied 
Psychology . October 195 $), pp. 287-292. 



The article summarizes five studies designed to compare the 
effects of a single versus repeated role playing experiences 
upon foremen. Three groups of trainees received one session 
while two groups received four sessions. Trainees answered 
open-ended questions on a before -and -after basis. Responses 
were rated on an Employee -Orientation scale and a Sensitivity 
scale. 



I 



The results disclosed no significant difference in scores between 
the groups that had had a single session and the groups that had 
repeated exposure to role playing. The authors found also that 
changes on scales occurred only if role playing was emotionally 
meaningful to the participant. 



17. Mahoney, Thomas A., Thomas H. Jerdee, and Abraham Korman, "An Experimental 

Evaluation of Management Development," Personnel Psychology , Spring i960 , pp. 81-98, 



This study was designed to measure changes produced by a 
management development program which relied heavily on the 
case study technique. There were four experimental groups 
of 12-15 supervisors each and one control group. Criterion 



measures, administered on a pre- and post -training basis, 



included a quiz testing knowledge of concepts and principles 
taught; written analyses of case material; and attitude scales 
to measure appreciation and sense of personal responsibility 
for self -development . 



1 



The case analyses and the attitude scales yielded statistically 
significant differences in the scores of the experimental and 
control groups but the quiz did not. The authors concluded 
that, on the whole, the training technique did not accomplish 
the desired objectives. 
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l8. Moffie, Dannie J., Richard Calhoon, and James K. O'Brien, "Evaluation of a 

Management Development Program," Personnel Psycholog y, Winter pp. 431-440. 

For abstract, see item 4. 

19* Morton, Robert B., and Bernard M. Bass, "The Organizational Training Laboratory," 
Training Directors Journal , October pp. 2-15. 

For abstract, see item 5. 

Oshry, Barry I., and Roger Harrison, "Transfer from Here-and-Now to There-and- 
Then: Changes in Organizational Problem Diagnosis Stemming from T-Group 

Training," Journal of Applied Behavioral Science , Apr il/May/ June PP* 185 -198. 

Report of a study to determine whether participants in a T-group 
can transfer their training experience to their work world. Data 
were collected from 46 middle managers before and after a two -week 
residential human relations laboratory program. The evaluation 
instrument was a Problem Analysis Questionnaire which measured 
various aspects of participants ' diagnostic approaches to inter- 
personal work problems. The problems were actual situations faced 
by the subjects on their jobs, and the questionnaire measured 
their perception of causes, resources available for coping, and 
their own ability to control or change the problem situation. 

The authors found that diagnostic orientations learned about self 
in relation to the T-group appear to generalize to learning about 
self in relation to work. After training, the participant (l) had 
an increased sense of personal control over the work situation; 

(2) was more aware of his own and others' personal and interpersonal 
needs;' ( 3 ) saw a clearer connection between how well interpersonal 
needs are met and how well the work gets done. However, he did not 
see a clear connection between his new perceptions and ways of 
translating them into action on his return to the job. 



Behavior 

"Did the training lead to more effective perfomiance on the job?" 

21. Argyris, Chris, "Explorations in Interpersonal Competence - I," Journal of 
Applied Behavioral Science January/February/March pp. 58 - 83 . 

A set of categories to measure interpersonal competence of participants 
of group efforts was developed, tested, and then used to measure 
the effects of T-group attendance. The sample consisted of 51 
members of four T-groups. Participants were assigned competence 
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scores and ranked according to their scores. Observers were 
then asked, at the end of the program, to rank each individual, 
and these rankings were compared with the competence scores. 

Agreement between scores and rankings was significant at the 
.05 level. 

Twenty-one participants evaluated bhe T-group in terms of 
their satisfaction with the experience. For 17 of the 21, 
there was high correlation between the quantitative scores, 
on the one hand, and participants * perceived degree of learning 
and their satisfaction with the laboratory . experience, on the 
other. 

22. Boyd, J. B. , and J. D. Elliss, Findings of Research into Senior Management 
Seminars. Hydro-Electric Power Commission of Ontario, Toronto, c '1962} 

This study was designed to measure behavior change as a result of 
T-group attendance. It compared the effects of laboratory 
training with the effects of a program built around case 
discussions and lectures. The sample consisted of managers of 
a Canadian utility. Estimates of change were obtained through 
interviews with the supervisor, two peers, and two subordinates 
of each participant six weeks and six months after the completion 
of training. A non-trained control group was used. 

The results showed that the observers of the laboratory parti- 
cipants reported the greatest amount of change, and the observers 
of the nontrained managers reported the smallest amount of change. 

23. Buchanan, Paul C., and Philip H. Brunstetter, "A Research Approach to Management 

Improvement: II. Methodology and Results of Evaluation," Journal of the 

American Society of Training Directors , February ^195^.^ pp • 18-27 • ~ 

This is an account of a study designed to evaluate the effectiveness 
of an intraorganizational laboratory program directed at organization 
development. Managers in one department (N=224) were the 
experimental group and managers in another .department (N=133) served 
?ls the control group. A post-training questionnaire was adminis- 
tered three to seven months after completion of the program to 
subordinates of the participants . They were asked to indicate 
changes which had occurred in the operation of their organizational 
units during the previous year. 

The findings were that the experimental group reported a greater 
number of changes in the direction of improvement than the control 
group . 
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24. Bunker, Douglas R., "Individual Applications; of Laboratory Training," Journal 
of Applied Behavioral Science , April/May/June pp. 131-148. 

The author describes a study of the effects of a T-group 
experience on the job behavior of participants . The sample 
consisted of 346 people at six different training laboratories 
conducted at Bethel, Maine, in i960 and 1961. Methodology 
included: a matched-pair control group obtained by participant 

nomination; open-ended behavior change descriptions obtained 
from the trainee and two or more co-workers; an objective 
coding system, which increased scoring reliability and permitted 
an assessment of the content of the components in each subject’s 
total change score. 

The results showed that trainees were seen by co-workers as 
increasing significantly more than controls in cognitive openness, 
behavioral skill, and understanding of social processes. On-the- 
job effects of training were found to correlate significantly with 
learning measures taken at the time of training. 



25. /Bunker, Douglas R., and Eric S. Knowles, "Comparison of Behavioral Changes 
Resulting from Human Relations Training Laboratories of Different Lengths," 
Journal of Applied’ Behavioral Science , October /November/ December <1967.^ 

pp . 505-524. 




The objective of this study was to assess the effect of duration 
of training on amount of behavioral change in a work setting. 

Study samples participated in human relations training labora- 
tories of three weeks' and two weeks' duration. A matched control 
group was used. The research instrument was a questionnaire 
administered eight to ten months after training to elicit descrip- 
tions of trainees' post -lab oratory behavior change, as perceived 
by the trainee himself and seven of his co-workers (superiors, 
subordinates, and peers). Two interrelated measures of change 
were derived: a "total change score," composed of the total number 

of different changes mentioned, and a "verified change score," 
composed of those behavioral changes which were mentioned by two 
or more persons . A set of 17 inductively-derived content categories 
4 were also used to make a qualitative analysis of the changes. 



The results showed that both laboratory -trained samples differed 
from the control sample on both measures ; they also differed 
significantly from each other in amount and kind of change. The 
three-week sample made more changes, as indicated by the two scores; 
changes were in the direction of more pro-active and interactive 
'behavior, as indicated by the content analysis. Occupational groups 
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seemed to be differentially responsive to different -length 
laboratories. The authors conclude, however, that differences 
in the training designs of the three -week and two -week labora- 
tories confound the duration variable. 



'riedlander, Frank, "The Impact of Organizational Training Laboratories upon 
he Effectiveness and Interaction of Ongoing Work Groups," Personnel Psychology , 
Autumn 1967? pp. 289-308. 



The purpose of the research project was to study the impact of 
organizational training laboratory sessions on members of four 
work groups (N-31) . Eight similar groups (N=60) which did not 



participate served as a "comparison" group. Criteria were six- 



factored dimensions, each composed of items which group members 
had perceived as problems in the course of a series of interviews . 
A questionnaire (Group Behavior Inventory) was administered 
before and six months after training. 



Significant changes were found in the trained groups on three 
dimensions: group effectiveness, mutual influence, and personal 

involvement; no significant changes occurred in leader approach- 
ability, intragroup trust, and evaluation of group meetings. 



27 , 



Harmon, Francis L., and Albert S. Glickman, "Managerial Training: Reinforcement 

Through Evaluation," Public Personnel Review , October 3.^65, pp. 194-196. 



Two sets of open-ended questionnaires, one for trainees and the 
other for their supervisors, were administered before, immediately 
after, and 8-10 months following completion of a training course 
in problem -analysis and decision-making. Participants were 5^- 
employees in middle and upper management positions in the U. S. 
Department of Agriculture. 



The responses showed (l) that methods of administering the program 
could be improved, and (2) that follow-up encourages participants 
to apply training to their work. The authors feel that this 
approach to evaluation can be used to trace • changes in the 
effectiveness of the training program. By pinpointing weaknesses, 
the procedure can lead to better programs. The evaluation process 
can also be used to promote positive reinforcement of what has 
been learned. 



28, 



Malouf, LeRoy G., "Managerial Grid Evaluated," Training and Development Journal . 
March 19®? pp. 6-15 • 



This is a report of a study designed to assess changes resulting 
from participation in a 1-week Managerial Grid management 
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development program. Eleven trainees were given a set of 
questions which they answered before participation, immediately 
after, and five months after completion of the sessions. 
Questionnaires were also sent to trainees' subordinates and 
bosses. 



2 



The results disclosed significant improvement in terms of 
human relationships, decision-making ability, etc., both on 
the post-test and after five months. The comments of subordi- 
nates corroborated the findings but the changes were less 
apparent to the trainees' superiors. 



1 



29. Miles, M. B. , "Changes During and Following Laboratory Training: A Clinical- 

Exper iment al Study , " Journal of Applied Behavioral Science , July/August/ 
September /19§5> PP* 215-242. 

'•WjKO*’- 



This study sought to evaluate a human relations training 
laboratory in terms of behavior change in the organizational 
setting. The experimental group was composed of 34 elementary 
school principals; two other groups of principals served as 
controls. A large variety of measures were obtained before, 
during, and after training from the trainees, their work 
associates, and the trainers. Instruments included structured 
and open-ended change -description questionnaires, rating of 
training behavior, and a series of personality measures. 



Results obtained with the perceived -change measure were 
statistically significant. The observers reported behavioral 
changes for a higher percentage of experimentals than for either 
of the control groups. Self-reported changes were also highest 
in the experimental group. Participants tended to report more 
changes than the observers. A content analysis of the changes 
reported revealed them to be mostly in the areas of increased 
sensitivity to others, greater communication and leadership 
skills, and more relaxed attitude on the Job. 



30. Merton, Robert B., and Bernard M. Bass,.J'The Organizational Training Laboratory," \ 
Training Directors Journal . October 1$64, 1 pp. 2-15. 



For abstract, see item 5* 



31 < 



Schutz, William, and Vernon L. Allen, "The Effects of a T-Oroup Laboratory on 
Interpersonal Behavior," Journal of Applied Behavioral Soionoo > July/August/ 
September 1966 $ PP* 265-200. 



The authors describe a study to investigate the effects of the 
T-group process at a Western Training Laboratory in Human 
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Relations. Sixty-eight participants completed the FIRO-B 
(Fundamental Interpersonal Relations Orientation - Behavior) 
questionnaire before and after the 2-week laboratory and 
again six months later. They also answered a series of open- 
ended questions on their perceptions of the effects, both 
positive and negative, of the T-group on them. A group of 
college students served as control group. 

The findings indicated significant changes in interpersonal 
behavior following the WLT experience. Correlations among 
before, after and six months later administrations of the 
questionnaire revealed much greater change in the WTL 
participants than in the control group. A content analysis 
of the subjective material showed that participants reported 
substantial increase in intellectual understanding and 
enhancement of personal relations in terms of less tension 
and more honesty, self -acceptan.ee and assertiveness. A large 
majority of changes persisted or grew after six months. 

32. Schwarz, Fred C., William P. Stilwell, and Burt K. Scanlan, "Effects of 

^Management Development on Manager Behavior and Subordinate Perception," Training 
\Y and Development Journal , April 1968, pp. 38-50; May 1968, pp. 24-30. 

The two-part article describes a research project designed to 
(l) assess the impact of a series of management development 
seminars, conducted at a large midwestern insurance company, 
on the behavior of participants, and (2) identify subordinates' 
reactions to any changes that did occur. Participants were 57 
managers representing the top five levels of the organization 
and all functional areas . Data were collected via the Leader 
Behavior Description Questionnaire (LBDQ) and open-ended 
interviews during which participants described "critical incidents." 

The managers were divided into two groups ; one was interviewed 
after training and one was interviewed before training. 

Analysis of subordinates' reactions to change in executive 
behavior, as measured by the LBDQ, showed that the subordinates 
of one group of managers perceived them as less active, less 
definite and less product ion -centered after training whereas the 
subordinates of the other group perceived their superiors in 
precisely the opposite way, more active, more product ion -oriented, 
etc. 

Analysis of the critical incidents showed that the trained group 
liras more personnel -oriented than the untrained group and more 
effective in solving problems. The trained group also showed 
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a much greater tendency to learn from day-to-day experience. 

33. Underwood, W. J..,. "Evaluation of Laboratory Method Training," Training Directors 
Journal , May (19§5 > PP* 34-40. 

The objective of the study was to measure the effects of 
T-group training in terms of changes in behavior as reported 
by observers selected by the trainees . The sample consisted 
of 15 supervisors who participated in 30 hours of in-plant 
T-group training. The control group consisted of 15 super- 
visors who were matched with the experimentals on department, 
organization level, and age. Observers were instructed to 
report anonymously any changes they perceived in the trainee's 
characteristic behavioral pattern. The observers coded each 
change according to whether it resulted in more or less 
effective supervision. The observation period covered the 
15 weeks of the course and extended 15 weeks beyond the end 
of the course. 

The findings were that (l) the trainees produced more observable 
changes in job behavior than the control group but (2) the 
changes were judged to be in the direction of decreased effec- 
tiveness, in a substantial number of reports. 

34. Wilson, John E., Donald P. Mullen, and Robert Morton, "Sensitivity Training for 
y Individual Growth - Team Training for Organization Development?" Training and 

v V Development Journal , January 1968, pp. 47-53* 

^ " For abstract, see item 7* 



Results 

"What were the effects of the training program 
as indicated by objective criteria such as 
'productivity, turnover, absenteeism, etc.?" 

35. Blake, R. R. , J. S. Mouton, L. B. Barnes, and L. E, Greiner, "Breakthrough in 
Organization Development," Harvard Business Review , November/Deceraber 1964* 
pp. 133-155* 

This article describes the administration and evaluation of an 
organization development program using Managerial Grid material. 

The primary objective of the research effort was to determine 
■whether the program had been successful in transferring behavioral 
science concepts into organizational action. Before -and -after 
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measures included indices of net profit, productivity 
per employee, controllable costs, etc. Anecdotal and test 
data were also gathered from participants and subordinates 
after completion of the program. 

The data indicated that the program made important 
contributions to improving productivity and profits as a 
result of changes in. behavior, attitudes, and values. 

The author® conclude that behavioral science education 
can assist large-scale organization development provided 
that the top management group is enthusiastic and involved, 
the educational strategy effectively builds team problem 
solving, and boss and colleagues reinforce the manager’s 
efforts to change his behavior. 

36. Hillman, Harry A., "Measuring Management Training - A Case Study." Journal 
of the Anerican Society of Training Directors » March 1962$' pp. 27-31* 

Thi ti study attempted to assess the effectiveness of a 
training program in terms of turnover, absenteeism, 
accidents. Workmen Compensation costs, etc. Other evalua- 
tion measures were also used, including (l) before-and- 
after tests on supervisory practices; (2) structured 
quest ionnaire on which trainees graded, on a 5 -point scale, 
the conference leader’s effectiveness, the interest value 
of the subjects, and the usefulness and applicability of 
the material; (3) form soliciting comments and suggestions 
from the trainees. 

With respect to the operational effects of the program, 
the findings were that turnover was 3<$ and the accident 
rate was 50 fo below the respective prior 5-year averages. 

Workmen Compensation costs and absenteeism were also 
reduced. The author cautions that it is not known to 
what extent improved operations were due to the training 
as compared to other factors; at the same time, he feels 
data justify the conclusion that a well-planned industrial 
program including management training can contribute in a 
material way to enhancing organizational efficiency. 
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Parker, Treadway C . , The Anatomy of Organization Development . New York: 
American Foundation for Management Research, Inc . , 1968, 163 pp. 

This report describes an exploratory study of a planned 
organization development program conducted within a large 
food products company. The study had several objectives, 
one of which was to observe and assess the effects of the 
change effort upon members of the company' s management . 

The formal development program, which wai3 designed to 
diffuse a set of management concepts throughout the 
entire organization, consisted of a series of briefings 
attended by three levels of management in a proscribed 
sequence: first, by the president, then by the 

corporate officers, and finally by the middle manage- 
ment group. In all, about 300 msoiagers attended the 
briefings. The study was longitudinal in nature, 
extending over a period of approximately !§• years. 

A series of semi -structured interviews were conducted 
with approximately 35 executives before, during, and 
after their attendance at the briefings. 



The major effects *of the program in terms of organi - 
zational performance were: (l) During the period of 
the development program, the growth rate for sales was 
almost double the rate for the prior eight years; (2) 
interest in external management development activities 
increased from an average annual enrollment of 36 in 
prior years to an average enrollment of 135 per year 
during the development period. 



The major effects of the program in terms of organi - 
zational climate were: (l) More emphasis was placed on 
delegating decision-making authority; (2} major changes 
occurred in attitudes toward planning and in actual 
planning practices; (3) as decision autonomy and clearer 
plans emerged, the need for a clearer concept of manage- 
ment control or results measurement became obvious; (4) 
a more homogeneous communication framework developed 
among the participating managers at all levels; ( 5 ) as 
a result of preparing job descriptions, there was a 
marked increase in clarity both about individual jobs 
and the interrelations among work groups; (6) with 
clearer job objectives, a new sense of objectivity 
regarding performance evaluation began to emerge. 
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II. Reviews of the Literature 



38, 



Besco, Robert, Joseph Tiffin, and Donald C. King, "Evaluation Techniques for 
Management Development. .Programs , " Journal of the American Society of Training 
Directors, October ’1959'/ PP* 13-27. 



A comprehensive literature survey of material published between 
1948 and 1958 on every aspect of evaluation: theory, 

methodology, interpretation of research findings, etc. An 
extensive bibliography is provided. 



39. 



Blumenfeld, Warren S., "Attitude Change as a Criterion in Training," Training 
and Development Journal , October £[9^/ PP* 26-32. 



A survey of studies (1957-61 ) which use attitude change as a 
criterion for evaluating the effectiveness of training. Nineteen 
studies are reviewed and classified according to the nature of 
the research design; only eight had a design calling both for 
pre- and post -tests and use of control group, and the author feels 
that this is cause for concern. 




Campbell, John P., and Marvin D. Dunnette, "Effectiveness of T-Group Experiences 
in Managerial Training and Development," Psychological Bulletin , August 1968, 
pp. 73-iou. 




A detailed examination of more than 40 studies which investigated 
the utility of the T-group as an educational technique. The 
authors also discuss the problem facing T-group researchers and 
offer suggestions for new approaches. The article contains 94 
references . 



ouse, Robert J., "T-Group Education and Leadership Effectiveness: A Review 

of the Empiric Literature and a Critical Evaluation," Personnel Psychology , 
Spring 1967, pp. 1-32. 



This article reviews studies concerned with the effects of T-groups 
in terms of (l) changes in participant characteristics, and (2) 
changes in behavior on the job. Author also discusses a series of 
questions which should be considered before the T-group technique 
is selected as a method of change within an organizational 
framework. Article lists 49 references. 



42, 



Miraglia, Joseph F., "Human Relations Training: A Critical Examination of 

On-the-Job Effects," Training and Development Journal , September Q.966 ,/ pp. 18-27 . 
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A review of the literature focusing on studies (1951-64) 
which attempted to evaluate human relations training in terms 
of on-the-job performance. The author concludes that most of 
them were poorly executed and attributes inadequacies to (l) 
lack of valid and reliable means of measurement; (2) 
unwillingness of organizations to experiment in sensitive 
human relations areas; (3) inherent difficulties of investi- 
gating and experimentally controlling dynamic interaction 
within organizations; and (4) difficulty of evaluating 
training results in ever-changing organizational settings. 



III. Discussions of Evaluation Techniques 




'enova, Charles C., "Is This Any Way to Evaluate a Training Activity?” Personnel 
Journal, July 1968, pp. 488-493* 



The author suggests tests to evaluate three aspects of a 
training program: degree of change in employee behavior; 

impact of training on achievement of organizational goals ; 
and effectiveness of training personnel, methods, and 
techniques. He lists different types of tests and briefly 
gives the advantages and disadvantages of each. 



44. J?erguson, Wilburn C., "Quantitative Evaluation of Training Using Student 
\ I^^R'eaction," Training and Development Journal , November 1968, PP* 36-42. 

"I \ A description of a method for evaluating training courses at the 

Motorola Aerospace Center by using trainee ratings, absentee 
rate, and dropout rate as criteria for determining reaction to 
the programs. Sample forms and instructions for calculating 
ratings sire included. 



45. Hennessy, David E., "Profit from Training," Training in Business and Industry , 
February 1969, PP* 34 -40. 




Guidelines for a method of measuring the effectiveness of a 
training program in financial terms . Step-by-step procedures 
for developing and evaluating the training program are provided, 
including sample cash flow diagrams. The writer, however, admits 
the difficulty of applying his system to a management development 
program. 
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46. Kirkpatrick, Donald L., "Evaluation of Training." In Craig, Robert L., and 
Lester R. Bittel (Eds.), Training and Development Handbook . New York: 
McGraw-Hill, Inc., 19^7? pp. 85 -119 • 

The author identifies four major areas of training evaluation. 

In order of increasing complexity, these are: 

(1) Reaction - how satisfied was the participant with the 
program? 

(2) Learning - what attitudes, techniques, or skills were 
understood and absorbed by the participant? 

(3) Behavior - did the training lead to more effective 
performance on the job? 

(4) Results - what were the effects of the training program 
as indicated by objective criteria such as productivity, 
turnover, absenteeism, etc.? 

Each area is discussed in some detail, and criteria and procedures 

for conducting the evaluation are outlined. Sample questionnaires 

are also included. 

47. Kohn, Vera, and Treadway C. Parker, "Some Guidelines for Evaluating Management 
Development Programs," Training and Development Journal , (in press). 



The authors report on the methodological aspects of two 
evaluation studies to illustrate the use of a variety of 
measuring devices and analytical techniques. The description 
of the methodologies serves as a backdrop against which ideas 
are offered to assist those interested in evaluation on 
formulating a research strategy. 

48. Reddin, W. J., "The Blake Approach and the Grid - A^Critique,*' Journal of the 
American Society of Training Directors , December 1963? P* 34. 

A list of 17 questions is presented for use in evaluating 
the Managerial Grid management development program; also, 
five criteria for comparing the Blake, case study, and T-group 
approaches to management development. 




! 



49. 



Rose, Homer C., "A Plan for Training Evaluation," Training and Development 
Journal, May 1968, pp. 38-51. 




The author discusses the purposes of evaluation; what should 
be evaluated (namely, the training plan as related to well 
established needs; methods, instructional materials and 
training aids; performance and achievement of trainees). He 
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stresses the importance of having those who are to do the 
evaluating participate in the planning of it. A number of 
l sample evaluation plans are provided, and thirteen techniques 

of evaluation are reviewed briefly. 

50. Shaffer, Dale E., ’’Control Through Measurement: Meeting Objectives as a Means 

f to Justify Training," Training Directors Journal , S ept ember pp. 39-50. 

| A discussion of the importance of exercising control over 

l training programs by (l) getting trainee opinions, and (2) 

E measuring learning through pre- and post -tests. As control 

l devices, the author prefers these methods to later follow-up, 

e.g., measuring changes in on-the-job behavior. 

f 51. Wnuk, Joseph J., Jr., "Evaluation. of Conceptual Training," Training and 

f, Development Journal , December (19^/ PP- 38-40. 



Brief account of a technique for determining the impact of 
a training program using conceptual material. Three factors 
are tapped on a pre- and post-training basis: awareness of, 
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IV. Surveys of Corporate Practices 

52. Catalanello, Ralph F. , and Donald L. Kirkpatrick, "Evaluating Training Programs - 
The State of the Art," Training and Development Journal , May 1968, pp. 2-9. 

j The authors surveyed 154 firms to determine the extent to which 
organizations which offer human relations training programs 
''i approach evaluation from each of the four vantage points 

identified by Kirkpatrick (see item 45) . They found that 77$ 
of the responding organizations assessed their programs in 
terms of trainees' reactions; 50$ attempted to measure learning; 

54$ studied changes in on-the-job behavior; and 45$ examined 
.results,, The authors conclude that "the evaluation 'state of 
the art’ is still in its infancy." 



53* Lippitt, Gordon L., Shirley D. McCune, and Larry D. Church, "Attitudes of Training 
Directors Toward the Application of Research to Training Programs," Training 
Directors Journal , March 'XS$4, pp . 13-20. 

V 

' This article reports the results of a questionnaire survey 
:aade among training directors to determine to what extent 
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they evaluate their programs. The post-training questionnaire 
is the most frequently-used instrument for evaluation. Most 
of the. respondents said they were not satisfied with the 
methods they use but can’t do more for lack of time, staff, and 
money. The most frequent application of research findings is 
toward improvement of program design. 



V . Books 



54 . Argyris, Chris, Interpersonal Competence and Organizational Behavior . Homewood, 

111.: Richard D. Irwin, Inc . ,Q9o2). 

55. Bradford, L. P., J. R. Gibb, and K. D. Benne, (Eds.) T -Group Theory and 

Laboratory Method . New York: John Wiley & Sons, Inc . , (j,96 jf . 



. 56 i-,- -Campbell, John P. , Management Training: The Development of Managerial 

A'V Effectiveness . Greensboro, N. C. : The Richardson Foundation, 1966 . 






57. 



Hesseling, P. , 
Company, N.V. , 



Strategy of Evaluation Research . 



Netherlands: Van Gorcum & 




House, Robert J., Management Development: Design, Evaluation and Implementation . 

Ann Arbor: Bureau of Industrial Relations, University of Michigan, 19&7. 



59 • McGehee, William, and Paul W^ Thayer, Training in Business and Industry . New York: 
John Wiley & Sons, Inc., 3 ^j 6 jL) 

60 . Miner, John^B. , Studies in Management Education . New York: Springer Publishing 

Company, ^ 965 ) 

- ***•*♦■— 

Tracey, William R., Evaluating Training and Development Systems . New York: 

American Management Association, 1968 . 
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